Extract metadata table and add schema editor preview by kdhillon-stripe · Pull Request #142 · stripe/sync-engine-fork

kdhillon-stripe · 2026-03-12T03:16:56Z

Most changes are visual, describing below:

Add a new /visualizer package. Client side playground for exploring the schema.
Artifact generation scripts in (scripts/explorer-*) used during build to create snapshots for the in browser data.
Sync Engine: Renaming sync metadata table accounts to _sync_accounts to avoid collisions with user-facing "accounts" Resource. Also renamed functions interacting with this table to clearly denote that this is a sync managed table.

Running

Generate Data: Run pnpm explorer:build to create the mock Stripe schema and seed data.
Start App: Run pnpm visualizer and navigate to http://localhost:3001.
Explore: Select tables from the sidebar or write custom SQL in the editor to query the local database.

Future

Add ERD
Remove /dashboard entirely

Copilot

Pull request overview

Adds a standalone client-side schema visualizer and supporting explorer build pipeline, while renaming the sync-owned metadata accounts table to _sync_accounts to avoid collisions with the projected Stripe accounts resource table.

Changes:

Introduces packages/visualizer (Next.js + PGlite) for in-browser SQL exploration of generated Stripe schema/data.
Adds scripts/explorer-* pipeline (Docker Postgres harness → migrations → deterministic seed → export artifacts).
Updates sync-engine migrations/DB APIs/tests to use _sync_accounts and adds OpenAPI migration tableMode support (runtime_required vs all_projected).

Reviewed changes

Copilot reviewed 39 out of 42 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
scripts/explorer-seed.ts	Deterministic seeding for core + long-tail tables; inserts sync account root and projected data.
scripts/explorer-migrate.ts	Runs migrations for explorer DB, enabling `all_projected` mode.
scripts/explorer-harness.ts	Docker Postgres harness with safety checks and `.tmp` metadata.
scripts/explorer-export.ts	Exports catalog-derived DDL + data into `bootstrap.sql` and writes `manifest.json`.
scripts/explorer-build.ts	Orchestrates harness → migrate → seed → export → cleanup.
scripts/README.md	Documentation for explorer scripts and workflows.
scripts/EXPLORER_RUNBOOK.md	Runbook for generating artifacts and running/deploying the visualizer.
packages/visualizer/tsconfig.json	TypeScript configuration for the visualizer Next app.
packages/visualizer/src/lib/pglite.ts	PGlite hydration hook (manifest + bootstrap SQL/JSON).
packages/visualizer/src/app/page.tsx	Client-only entry loading `ExplorerClient` with a skeleton.
packages/visualizer/src/app/layout.tsx	App layout + metadata.
packages/visualizer/src/app/globals.css	Tailwind import for global styling.
packages/visualizer/src/app/ExplorerClient.tsx	Main UI: table list, SQL editor, results grid, pagination, resizing.
packages/visualizer/postcss.config.mjs	PostCSS config for Tailwind v4 plugin.
packages/visualizer/package.json	Visualizer dependencies (Next/React/PGlite/CodeMirror) and scripts.
packages/visualizer/next.config.ts	Enables WASM + COOP/COEP headers for PGlite.
packages/visualizer/next-env.d.ts	Next TypeScript ambient references.
packages/visualizer/README.md	Visualizer package documentation.
packages/visualizer/.gitignore	Ignores generated explorer artifacts and build outputs.
packages/sync-engine/src/types.ts	Updates docs to mention `_sync_accounts` as sync metadata table.
packages/sync-engine/src/tests/testSetup.ts	Uses `upsertSyncMetadataAccount` helper for FK root.
packages/sync-engine/src/tests/integration/postgres-sync-observability.test.ts	Adds coverage for new sync-account helpers + scoped deletion.
packages/sync-engine/src/tests/e2e/account-management.e2e.test.ts	Updates E2E flows to `_sync_accounts` and new deletion helpers.
packages/sync-engine/src/stripeSync.ts	Switches to sync-metadata account read/upsert APIs.
packages/sync-engine/src/openapi/specParser.ts	Allows “all projected” parsing when `allowedTables` is omitted.
packages/sync-engine/src/openapi/postgresAdapter.ts	FK target becomes configurable and defaults to `_sync_accounts`.
packages/sync-engine/src/openapi/tests/specParser.test.ts	Tests new parsing modes + real-spec comparison.
packages/sync-engine/src/openapi/tests/postgresAdapter.test.ts	Updates FK expectations + adds projected `accounts` test.
packages/sync-engine/src/database/postgres.ts	Renames metadata account APIs + adds sync-account-scoped counting/deletion.
packages/sync-engine/src/database/migrations/0001_rename_sync_accounts.sql	Renames sync-owned `stripe.accounts` → `stripe._sync_accounts` and updates FKs.
packages/sync-engine/src/database/migrations-embedded.ts	Embeds the new migration 0001.
packages/sync-engine/src/database/migrate.ts	Exposes `MigrationConfig`, adds `tableMode`, updates FK to `_sync_accounts`.
packages/sync-engine/src/database/tests/migrate.tableMode.test.ts	Type-level coverage for `tableMode`.
packages/sync-engine/src/database/tests/migrate.openapi.test.ts	Updates baseline tables to `_sync_accounts`.
packages/sync-engine/scripts/test-table-modes.ts	Script to compare parsed table sets across modes.
packages/sync-engine/package.json	Version bump + repo/homepage URL updates + `@types/pg` bump.
packages/sync-engine/docs/table-mode-migration.md	Documentation for `tableMode` and parsing behavior.
packages/fastify-app/package.json	Bumps `@types/pg`.
package.json	Adds `explorer:build`, `visualizer`, and `visualizer:with-data` scripts; adds dev deps.
README.md	Mentions the new visualizer and workflows.
.gitignore	Ignores `.tmp/` and generated visualizer artifacts.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (2)

scripts/explorer-seed.ts:1

This script accesses DataGenerator's rng via this.gen['rng'], but rng is declared private on DataGenerator. In TypeScript this will fail type-checking/compilation. Prefer exposing the needed RNG operations as public methods on DataGenerator (e.g., nextTimestamp(), boolean(), choice(), nextInt()), or pass a SeededRandom into StripeDataGraph directly.
scripts/explorer-build.ts:1
On failure, the pipeline prints a generic 'Pipeline failed' but does not log the actual caught error, which makes local debugging and CI triage harder (especially when execPhase rethrows). Log error (or at least its message) here so the final output always includes the root cause after cleanup attempts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-12T03:40:51Z

+          const primaryKeyResult = await query(`
+            SELECT kcu.column_name
+            FROM information_schema.table_constraints tc
+            JOIN information_schema.key_column_usage kcu
+              ON tc.constraint_name = kcu.constraint_name
+              AND tc.table_schema = kcu.table_schema
+            WHERE tc.table_schema = 'stripe'
+              AND tc.table_name = '${tableName}'
+              AND tc.constraint_type = 'PRIMARY KEY'
+            ORDER BY kcu.ordinal_position
+            LIMIT 1
+          `)


Table names are interpolated directly into SQL strings (tc.table_name = '${tableName}' and FROM stripe.${tableName}), which allows SQL injection if tableName is ever influenced by untrusted data (and also breaks if identifiers need quoting). Use parameterized queries for values (e.g., tc.table_name = $1) and quote identifiers for table names (e.g., stripe.${quoteIdentifier(tableName)} for identifier contexts).

Copilot · 2026-03-12T03:40:51Z

+        tableOrderByColumnRef.current[tableName] = orderByColumn
+      }
+
+      const sqlText = `SELECT * FROM stripe.${tableName} ORDER BY ${quoteIdentifier(orderByColumn)} LIMIT ${pageSize} OFFSET ${offset}`


Table names are interpolated directly into SQL strings (tc.table_name = '${tableName}' and FROM stripe.${tableName}), which allows SQL injection if tableName is ever influenced by untrusted data (and also breaks if identifiers need quoting). Use parameterized queries for values (e.g., tc.table_name = $1) and quote identifiers for table names (e.g., stripe.${quoteIdentifier(tableName)} for identifier contexts).

Copilot · 2026-03-12T03:40:51Z

+-- Rename the sync-managed account root table to avoid collisions with the OpenAPi
+
+DO $$
+BEGIN
+  IF EXISTS (
+    SELECT 1
+    FROM information_schema.columns
+    WHERE table_schema = 'stripe'
+      AND table_name = 'accounts'
+      AND column_name = 'api_key_hashes'
+  ) AND NOT EXISTS (
+    SELECT 1
+    FROM information_schema.tables
+    WHERE table_schema = 'stripe'
+      AND table_name = '_sync_accounts'
+  ) THEN
+    ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";
+  END IF;
+END;
+$$;
+
+ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"
+  RENAME TO "idx_sync_accounts_api_key_hashes";
+
+ALTER TABLE "stripe"."_managed_webhooks"
+  DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";
+ALTER TABLE "stripe"."_managed_webhooks"
+  ADD CONSTRAINT "fk_managed_webhooks_account"
+    FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);
+
+ALTER TABLE "stripe"."_sync_runs"
+  DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";
+ALTER TABLE "stripe"."_sync_runs"
+  ADD CONSTRAINT "fk_sync_runs_account"
+    FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);


This migration hard-codes the stripe schema. The codebase now has schemaName/syncTablesSchemaName configuration (and the OpenAPI adapter takes accountSchema), so running migrations with a non-stripe sync schema would leave the metadata account table un-renamed and FKs un-updated. Consider parameterizing the schema (e.g., generating this SQL with the configured schema in runMigrationsFromContent), or implementing the rename in a TypeScript migration step that uses syncTablesSchemaName.

Suggested change

-- Rename the sync-managed account root table to avoid collisions with the OpenAPi

DO $$

BEGIN

IF EXISTS (

SELECT 1

FROM information_schema.columns

WHERE table_schema = 'stripe'

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = 'stripe'

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "stripe"."_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "stripe"."_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);

ALTER TABLE "stripe"."_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "stripe"."_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);

-- Rename the sync-managed account root table to avoid collisions with the OpenAPI.

-- This migration is written to work with the current schema, so it can be used

-- with configurable sync schema names (e.g., via search_path / current_schema()).

DO $$

DECLARE

schema_name text := current_schema();

BEGIN

-- Rename the accounts table to _sync_accounts if needed.

IF EXISTS (

SELECT 1

FROM information_schema.columns

WHERE table_schema = schema_name

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = schema_name

AND table_name = '_sync_accounts'

) THEN

EXECUTE format(

'ALTER TABLE %I.%I RENAME TO %I',

schema_name,

'accounts',

'_sync_accounts'

);

END IF;

-- Rename the index on api_key_hashes if it exists.

EXECUTE format(

'ALTER INDEX IF EXISTS %I.%I RENAME TO %I',

schema_name,

'idx_accounts_api_key_hashes',

'idx_sync_accounts_api_key_hashes'

);

-- Update the foreign key on _managed_webhooks to point at _sync_accounts.

EXECUTE format(

'ALTER TABLE %I.%I DROP CONSTRAINT IF EXISTS %I',

schema_name,

'_managed_webhooks',

'fk_managed_webhooks_account'

);

EXECUTE format(

'ALTER TABLE %I.%I ADD CONSTRAINT %I ' ||

'FOREIGN KEY (%I) REFERENCES %I.%I (id)',

schema_name,

'_managed_webhooks',

'fk_managed_webhooks_account',

'account_id',

schema_name,

'_sync_accounts'

);

-- Update the foreign key on _sync_runs to point at _sync_accounts.

EXECUTE format(

'ALTER TABLE %I.%I DROP CONSTRAINT IF EXISTS %I',

schema_name,

'_sync_runs',

'fk_sync_runs_account'

);

EXECUTE format(

'ALTER TABLE %I.%I ADD CONSTRAINT %I ' ||

'FOREIGN KEY (%I) REFERENCES %I.%I (id)',

schema_name,

'_sync_runs',

'fk_sync_runs_account',

'_account_id',

schema_name,

'_sync_accounts'

);

END;

$$;

Copilot · 2026-03-12T03:40:51Z

+    WHERE table_schema = 'stripe'
+      AND table_name = 'accounts'
+      AND column_name = 'api_key_hashes'
+  ) AND NOT EXISTS (
+    SELECT 1
+    FROM information_schema.tables
+    WHERE table_schema = 'stripe'
+      AND table_name = '_sync_accounts'
+  ) THEN
+    ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";
+  END IF;
+END;
+$$;
+
+ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"
+  RENAME TO "idx_sync_accounts_api_key_hashes";
+
+ALTER TABLE "stripe"."_managed_webhooks"
+  DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";
+ALTER TABLE "stripe"."_managed_webhooks"
+  ADD CONSTRAINT "fk_managed_webhooks_account"
+    FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);
+
+ALTER TABLE "stripe"."_sync_runs"
+  DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";
+ALTER TABLE "stripe"."_sync_runs"
+  ADD CONSTRAINT "fk_sync_runs_account"
+    FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);


This migration hard-codes the stripe schema. The codebase now has schemaName/syncTablesSchemaName configuration (and the OpenAPI adapter takes accountSchema), so running migrations with a non-stripe sync schema would leave the metadata account table un-renamed and FKs un-updated. Consider parameterizing the schema (e.g., generating this SQL with the configured schema in runMigrationsFromContent), or implementing the rename in a TypeScript migration step that uses syncTablesSchemaName.

Suggested change

WHERE table_schema = 'stripe'

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = 'stripe'

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "stripe"."_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "stripe"."_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);

ALTER TABLE "stripe"."_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "stripe"."_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);

WHERE table_schema = current_schema()

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = current_schema()

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "_sync_accounts" (id);

ALTER TABLE "_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "_sync_accounts" (id);

Copilot · 2026-03-12T03:40:52Z

+-- Rename the sync-managed account root table to avoid collisions with the OpenAPi
+
+DO $$
+BEGIN
+  IF EXISTS (
+    SELECT 1
+    FROM information_schema.columns
+    WHERE table_schema = 'stripe'
+      AND table_name = 'accounts'
+      AND column_name = 'api_key_hashes'
+  ) AND NOT EXISTS (
+    SELECT 1
+    FROM information_schema.tables
+    WHERE table_schema = 'stripe'
+      AND table_name = '_sync_accounts'
+  ) THEN
+    ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";
+  END IF;
+END;
+$$;
+
+ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"
+  RENAME TO "idx_sync_accounts_api_key_hashes";
+
+ALTER TABLE "stripe"."_managed_webhooks"
+  DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";
+ALTER TABLE "stripe"."_managed_webhooks"
+  ADD CONSTRAINT "fk_managed_webhooks_account"
+    FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);
+
+ALTER TABLE "stripe"."_sync_runs"
+  DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";
+ALTER TABLE "stripe"."_sync_runs"
+  ADD CONSTRAINT "fk_sync_runs_account"
+    FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);


This migration hard-codes the stripe schema. The codebase now has schemaName/syncTablesSchemaName configuration (and the OpenAPI adapter takes accountSchema), so running migrations with a non-stripe sync schema would leave the metadata account table un-renamed and FKs un-updated. Consider parameterizing the schema (e.g., generating this SQL with the configured schema in runMigrationsFromContent), or implementing the rename in a TypeScript migration step that uses syncTablesSchemaName.

Suggested change

-- Rename the sync-managed account root table to avoid collisions with the OpenAPi

DO $$

BEGIN

IF EXISTS (

SELECT 1

FROM information_schema.columns

WHERE table_schema = 'stripe'

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = 'stripe'

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "stripe"."_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "stripe"."_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);

ALTER TABLE "stripe"."_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "stripe"."_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);

-- Rename the sync-managed account root table to avoid collisions with the OpenAPI

DO $$

DECLARE

schema_name text := current_schema();

BEGIN

-- Rename accounts -> _sync_accounts in the current schema if appropriate

IF EXISTS (

SELECT 1

FROM information_schema.columns

WHERE table_schema = schema_name

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = schema_name

AND table_name = '_sync_accounts'

) THEN

EXECUTE format('ALTER TABLE %I.accounts RENAME TO _sync_accounts', schema_name);

END IF;

-- Rename the index on api_key_hashes, if it exists, in the current schema

EXECUTE format(

'ALTER INDEX IF EXISTS %I.idx_accounts_api_key_hashes RENAME TO idx_sync_accounts_api_key_hashes',

schema_name

);

-- Update managed webhooks FK to point at _sync_accounts in the current schema

EXECUTE format(

'ALTER TABLE %I._managed_webhooks DROP CONSTRAINT IF EXISTS fk_managed_webhooks_account',

schema_name

);

EXECUTE format(

'ALTER TABLE %I._managed_webhooks ' ||

'ADD CONSTRAINT fk_managed_webhooks_account ' ||

'FOREIGN KEY (account_id) REFERENCES %I._sync_accounts (id)',

schema_name,

schema_name

);

-- Update sync runs FK to point at _sync_accounts in the current schema

EXECUTE format(

'ALTER TABLE %I._sync_runs DROP CONSTRAINT IF EXISTS fk_sync_runs_account',

schema_name

);

EXECUTE format(

'ALTER TABLE %I._sync_runs ' ||

'ADD CONSTRAINT fk_sync_runs_account ' ||

'FOREIGN KEY (_account_id) REFERENCES %I._sync_accounts (id)',

schema_name,

schema_name

);

END;

$$;

Copilot · 2026-03-12T03:40:52Z

+    WHERE table_schema = 'stripe'
+      AND table_name = 'accounts'
+      AND column_name = 'api_key_hashes'
+  ) AND NOT EXISTS (
+    SELECT 1
+    FROM information_schema.tables
+    WHERE table_schema = 'stripe'
+      AND table_name = '_sync_accounts'
+  ) THEN
+    ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";
+  END IF;
+END;
+$$;
+
+ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"
+  RENAME TO "idx_sync_accounts_api_key_hashes";
+
+ALTER TABLE "stripe"."_managed_webhooks"
+  DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";
+ALTER TABLE "stripe"."_managed_webhooks"
+  ADD CONSTRAINT "fk_managed_webhooks_account"
+    FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);
+
+ALTER TABLE "stripe"."_sync_runs"
+  DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";
+ALTER TABLE "stripe"."_sync_runs"
+  ADD CONSTRAINT "fk_sync_runs_account"
+    FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);


This migration hard-codes the stripe schema. The codebase now has schemaName/syncTablesSchemaName configuration (and the OpenAPI adapter takes accountSchema), so running migrations with a non-stripe sync schema would leave the metadata account table un-renamed and FKs un-updated. Consider parameterizing the schema (e.g., generating this SQL with the configured schema in runMigrationsFromContent), or implementing the rename in a TypeScript migration step that uses syncTablesSchemaName.

Suggested change

WHERE table_schema = 'stripe'

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = 'stripe'

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "stripe"."accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "stripe"."idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "stripe"."_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "stripe"."_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "stripe"."_sync_accounts" (id);

ALTER TABLE "stripe"."_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "stripe"."_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "stripe"."_sync_accounts" (id);

WHERE table_schema = current_schema

AND table_name = 'accounts'

AND column_name = 'api_key_hashes'

) AND NOT EXISTS (

SELECT 1

FROM information_schema.tables

WHERE table_schema = current_schema

AND table_name = '_sync_accounts'

) THEN

ALTER TABLE "accounts" RENAME TO "_sync_accounts";

END IF;

END;

$$;

ALTER INDEX IF EXISTS "idx_accounts_api_key_hashes"

RENAME TO "idx_sync_accounts_api_key_hashes";

ALTER TABLE "_managed_webhooks"

DROP CONSTRAINT IF EXISTS "fk_managed_webhooks_account";

ALTER TABLE "_managed_webhooks"

ADD CONSTRAINT "fk_managed_webhooks_account"

FOREIGN KEY ("account_id") REFERENCES "_sync_accounts" (id);

ALTER TABLE "_sync_runs"

DROP CONSTRAINT IF EXISTS "fk_sync_runs_account";

ALTER TABLE "_sync_runs"

ADD CONSTRAINT "fk_sync_runs_account"

FOREIGN KEY ("_account_id") REFERENCES "_sync_accounts" (id);

Copilot · 2026-03-12T03:40:52Z

+  // Test all_projected mode (omit allowedTables per the interface documentation)
+  const allProjectedSpec = parser.parse(resolvedSpec.spec, {
+    resourceAliases: OPENAPI_RESOURCE_TABLE_ALIASES,
+    // No allowedTables property - per types.ts: "If omitted, all resolvable x-resourceId entries are parsed"
+    // Note: Current implementation defaults to RUNTIME_REQUIRED_TABLES when omitted
+  })


The inline comments conflict with the updated SpecParser behavior in this PR (omitting allowedTables no longer defaults to RUNTIME_REQUIRED_TABLES). Please update/remove the stale comments so this script remains a reliable diagnostic for table-mode behavior.

@copilot make this change this feedback

Copilot · 2026-03-12T03:40:52Z

+      [...RUNTIME_REQUIRED_TABLES].sort()
+    )
+    expect(allProjectedParsed.tables.length).toBeGreaterThan(runtimeParsed.tables.length)
+    expect(allProjectedParsed.tables.length).toBe(106)


The test asserts an exact table count (106) when parsing the remotely-resolved Stripe spec. This is brittle because upstream spec changes (even for a fixed API version) can change the number of resolvable resource tables and cause unrelated CI failures. Prefer asserting relative properties (e.g., > runtime_required and includes some known additional tables) or setting a minimum threshold rather than an exact count.

Suggested change

expect(allProjectedParsed.tables.length).toBe(106)

@copilot make this greater than 100

Copilot · 2026-03-12T03:43:22Z

@kdhillon-stripe I've opened a new pull request, #144, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2026-03-12T03:43:50Z

@kdhillon-stripe I've opened a new pull request, #145, to work on those changes. Once the pull request is ready, I'll request review from you.

cla-assistant · 2026-03-12T03:48:11Z

All committers have signed the CLA.

Adds the Stripe schema visualizer (Next.js + PGlite) and explorer build pipeline from github.com/stripe/pull/142. - packages/visualizer: in-browser schema explorer powered by PGlite (WASM Postgres), with committed bootstrap.sql (106 tables, 2020-08-27 Stripe API schema with seed data) so the visualizer works immediately - scripts/explorer-{harness,migrate,seed,export,build}.ts: pipeline to regenerate bootstrap.sql for a new Stripe API version; note that explorer-migrate.ts requires the old monolith (not present in v2) - package.json: adds `visualizer` and `explorer:build` root scripts The accounts→_sync_accounts rename (also in PR #142) is not ported since the old sync_accounts table does not exist in v2's architecture. Run: pnpm visualizer (starts at http://localhost:3000) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

tonyxiao · 2026-03-25T09:17:21Z

Hey @kdhillon-stripe! We ported this PR onto the v2 branch — here's a summary of what was changed during the port:

Changes from v2 port

1. Moved to apps/ directory
packages/visualizer/ → apps/visualizer/ to match v2's convention where user-facing applications live in apps/.

2. Renamed package scope
@supabase/stripe-sync-visualizer → @stripe/sync-visualizer to match v2's @stripe/sync-* naming convention.

3. Fixed explorer-seed.ts imports
The seed script imported from packages/sync-engine which was decomposed in v2. Updated to import directly from packages/source-stripe/src/openapi/index.js and types.js.

4. Stubbed explorer-migrate.ts
runMigrations imported from the old monolith — stubbed with a clear error message since the migration path in v2 goes through packages/destination-postgres. This script needs a proper v2 implementation if anyone wants to regenerate bootstrap data.

5. Replaced pnpm tsx with bun run
tsx fails on our machines because esbuild's native binary doesn't work here. Updated all scripts and docs to use bun run instead.

6. Fixed explorer-export.ts output path
Output was hardcoded to packages/visualizer/public/explorer-data — updated to apps/visualizer/public/explorer-data.

7. Added pg to root devDependencies
The explorer scripts need pg but it wasn't resolvable from the repo root.

8. Added visualizer:with-data root script

"visualizer:with-data": "pnpm explorer:build && pnpm visualizer"

One-liner to regenerate explorer data and start the dev server.

The branch is at pr-142-on-v2 if you want to take a look.

Adds the Stripe schema visualizer (Next.js + PGlite) and explorer build pipeline from github.com/stripe/pull/142. - packages/visualizer: in-browser schema explorer powered by PGlite (WASM Postgres), with committed bootstrap.sql (106 tables, 2020-08-27 Stripe API schema with seed data) so the visualizer works immediately - scripts/explorer-{harness,migrate,seed,export,build}.ts: pipeline to regenerate bootstrap.sql for a new Stripe API version; note that explorer-migrate.ts requires the old monolith (not present in v2) - package.json: adds `visualizer` and `explorer:build` root scripts The accounts→_sync_accounts rename (also in PR #142) is not ported since the old sync_accounts table does not exist in v2's architecture. Run: pnpm visualizer (starts at http://localhost:3000) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

tonyxiao · 2026-03-25T10:07:17Z

One more round of changes on extract-metadata-table:

Removed all static files and build steps. pglite.ts now generates the Stripe schema entirely in-browser — no bootstrap.sql, no manifest.json, no explorer:build step.

How it works:

On first open, fetches https://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.json (~3 MB) directly in the browser
Runs SpecParser (already in the repo, zero Node deps) to discover all resource tables from x-resourceId fields
Builds CREATE TABLE IF NOT EXISTS DDL with an inline browser-compatible generator
Loads into PGlite — ready to query
Caches the generated SQL in sessionStorage so subsequent loads are instant

Files changed:

Deleted scripts/explorer-generate.ts, scripts/explorer-build.ts, scripts/explorer-harness.ts, scripts/explorer-seed.ts, scripts/explorer-export.ts, scripts/EXPLORER_RUNBOOK.md
Removed apps/visualizer/public/explorer-data/ entirely
Removed explorer:build / visualizer:with-data root scripts
Added @stripe/sync-source-stripe workspace dep to visualizer
Exported SpecParser, OPENAPI_RESOURCE_TABLE_ALIASES, ParsedResourceTable from source-stripe's public index

pnpm visualizer is now the only command needed to run it.

Adds the Stripe schema visualizer (Next.js + PGlite) and explorer build pipeline from github.com/stripe/pull/142. - packages/visualizer: in-browser schema explorer powered by PGlite (WASM Postgres), with committed bootstrap.sql (106 tables, 2020-08-27 Stripe API schema with seed data) so the visualizer works immediately - scripts/explorer-{harness,migrate,seed,export,build}.ts: pipeline to regenerate bootstrap.sql for a new Stripe API version; note that explorer-migrate.ts requires the old monolith (not present in v2) - package.json: adds `visualizer` and `explorer:build` root scripts The accounts→_sync_accounts rename (also in PR #142) is not ported since the old sync_accounts table does not exist in v2's architecture. Run: pnpm visualizer (starts at http://localhost:3000) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

- Move packages/visualizer → apps/visualizer (consistent with other apps) - Fix explorer-seed.ts: import from packages/source-stripe/src/openapi instead of deleted packages/sync-engine - Fix explorer-export.ts: output path apps/visualizer/public/explorer-data - Fix EXPLORER_RUNBOOK.md + apps/visualizer/README.md: replace pnpm tsx with bun run, update packages/visualizer refs to apps/visualizer - Add visualizer:with-data root script (explorer:build + visualizer) - Add pg to root devDependencies so explorer scripts resolve it Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

bootstrap.sql and manifest.json were generated files committed to the repo — requiring Docker + Postgres + stripe-mock to regenerate. Replace the entire pipeline (explorer-build/harness/migrate/seed/export) with a single script that generates CREATE TABLE DDL directly from the Stripe OpenAPI spec using the existing SpecParser + buildCreateTableWithSchema. - scripts/explorer-generate.ts: new 50-line script, runs in seconds - Remove explorer-build/harness/migrate/seed/export + EXPLORER_RUNBOOK - Gitignore public/explorer-data/bootstrap.sql and manifest.json - Simplify ExplorerManifest interface (drop seed/failedTables/verification) - Fix ExplorerClient.tsx to use manifest.tables[] instead of manifest.manifest{} - bootstrap.sql: 166 KB (was 1.1 MB with seeded data rows) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

Remove all static files and build steps. pglite.ts now fetches the Stripe OpenAPI spec from GitHub at runtime, runs SpecParser to get table definitions, and builds CREATE TABLE DDL directly in the browser. The spec (~3 MB) is cached in sessionStorage after the first load. - Delete scripts/explorer-generate.ts (no longer needed) - Remove public/explorer-data/ entirely (no static assets) - Remove explorer:build and visualizer:with-data root scripts - pglite.ts: inline browser-compatible DDL builder (no node: deps) - Add @stripe/sync-source-stripe workspace dep to visualizer - Export SpecParser, OPENAPI_RESOURCE_TABLE_ALIASES, ParsedResourceTable from source-stripe's public index Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

Remove stale pg/@types/pg devDeps from root package.json (leftover from removed explorer scripts). Restore v2's exact lockfile to avoid CI failures from peer-dep resolution differences, adding only the apps/visualizer workspace:* entry for @stripe/sync-source-stripe. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

Add all apps/visualizer workspace deps (Next.js, PGlite, CodeMirror, etc.) and accept peer-dep string updates from jiti@2.6.1 transitive resolution. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

source-stripe's main index.ts transitively imports specFetchHelper.ts which uses node:path — Next.js/webpack can't bundle node: URIs for client components. Add a ./browser subpath export pointing at a new browser.ts barrel that re-exports only SpecParser, OPENAPI_RESOURCE_TABLE_ALIASES, and ParsedResourceTable (all pure TS, no node: deps). Update pglite.ts to import from '@stripe/sync-source-stripe/browser'. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

kdhillon-stripe · 2026-03-25T16:35:11Z

@@ -0,0 +1,32 @@
+{
+  "name": "@stripe/sync-visualizer",


nit: we should call it @stripe/schema-visualizer

kdhillon-stripe changed the title ~~Extract metadata table~~ Extract metadata table and add visualizer Mar 12, 2026

kdhillon-stripe force-pushed the extract-metadata-table branch from df6d0eb to af159f8 Compare March 12, 2026 03:21

kdhillon-stripe requested a review from Copilot March 12, 2026 03:36

Copilot AI reviewed Mar 12, 2026

View reviewed changes

Copilot AI mentioned this pull request Mar 12, 2026

Remove stale SpecParser default comments from test-table-modes.ts #144

Merged

Copilot AI mentioned this pull request Mar 12, 2026

Use minimum threshold instead of exact count in specParser integration test #145

Merged

Yostra approved these changes Mar 17, 2026

View reviewed changes

tonyxiao force-pushed the extract-metadata-table branch from 1a51675 to 0186f22 Compare March 25, 2026 09:28

tonyxiao changed the base branch from main to v2 March 25, 2026 09:29

tonyxiao force-pushed the extract-metadata-table branch from 00b5790 to 1fe0f21 Compare March 25, 2026 10:09

tonyxiao and others added 8 commits March 25, 2026 04:51

Rename visualizer package to @stripe/sync-visualizer

7d891c9

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Committed-By-Agent: claude

tonyxiao force-pushed the extract-metadata-table branch from 1ca0e01 to 7ae4eab Compare March 25, 2026 11:51

kdhillon-stripe commented Mar 25, 2026

View reviewed changes

tonyxiao changed the title ~~Extract metadata table and add visualizer~~ Extract metadata table and add schema editor preview Mar 25, 2026

tonyxiao approved these changes Mar 25, 2026

View reviewed changes

tonyxiao merged commit 52cfe35 into v2 Mar 25, 2026
3 checks passed

tonyxiao deleted the extract-metadata-table branch March 25, 2026 16:42

Conversation

kdhillon-stripe commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Running

Future

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

kdhillon-stripe Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

kdhillon-stripe Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Mar 12, 2026

Uh oh!

Copilot AI commented Mar 12, 2026

Uh oh!

cla-assistant Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tonyxiao commented Mar 25, 2026

Changes from v2 port

Uh oh!

tonyxiao commented Mar 25, 2026

Uh oh!

kdhillon-stripe Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kdhillon-stripe commented Mar 12, 2026 •

edited

Loading

cla-assistant Bot commented Mar 12, 2026 •

edited

Loading